Edge superconducting state in attractive U Kane-Mele-Hubbard model 
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We theoretically investigate the phase transition from topological insulator (TI) to supercon- 
ductor in the attractive U Kane-Mele-Hubbard model with self-consistent mean field method. We 
demonstrate the existence of edge superconducting state (ESS), in which the bulk is still an insulator 
and the superconductivity only appears near the edges. The ESS results from the special energy 
dispersion of TI, and is a general property of the superconductivity in TI. The phase transition 
in this model essentially consists of two steps. When the attractive U becomes nonzero, ESS ap- 
pears immediately. After the attractive U exceeds a critical value U c , the whole system becomes 
a superconductor. The effective model of the ESS has also been discussed and we believe that the 
conception of ESS can be realized in atomic optical lattice system. 



The topological insulator (TI) has drawn a great deal 
of attention recently because it offers us a novel quantum 
state of electrons, i.e. topological insulating state which 
is insulating in the bulk but has edge states protected by 
time reversal symmetry^—. The topological insulating 
state results from its nontrival band topology induced by 
spin-orbit interaction and time reversal symmetry, and 
is characterized by a Zi topological invariant. The ex- 
perimental discovery of two-dimensional (2D) and three- 
dimensional (3D) TI in a variety of materials has greatly 
promoted the research interests in Tl^r— . Many intrigu- 
ing issues about TI has been proposed, for example, the 
realization of Majarona fermion in Ttf^, new kinds of 
spintronic or magnetoelectric device 9 - - — , and the strong 
correlation effects in Tli 2 - - — . 

The TI in strong correlation system mainly concerns 
two kinds of problem. One is about the electron-electron 
interaction induced topological insulating stated 2 - - — , and 
the other is about the novel phase transition between the 
topological insulating state and strong correlation quan- 
tum states^—. Kane-Mele-Hubbard model is the sim- 
plest and very important 2D strong correlation TI the- 
oretical modeli£~— . It basically is the Hubbard model 
on honeycomb lattice with spin-orbit coupling. The ex- 
otic strong correlation quantum states of the Hubbard 
model on honeycomb lattice, especially the spin liquid 
state, has been entensively studied recently 2 ^. Mean- 
while, the Kane-Mele model 2 ^, i.e. honeycomb lattice 
with spin-orbit coupling, describes the 2D topological in- 
sulating state, which is also named quantum spin hall 
(QSH) state due to the analogy to the quantum hall ef- 
fect. An interesting problem is what the effect of the 
interplay between the spin-orbit interaction and electron- 
electron interaction is. Recently, several works have been 
done to discuss the phase diagram of the Kane-Mele- 
Hubbard modeli£~— , including Quantum Monte Carlo 
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calculation*^— . Besides the theoretical interests, the 
QSH state has been proposed to be realized in various 
materials^— . 

Superconductivity in TI is also a research focus. On 
one hand, it is predicted that Majarona fermion can be 
realized on TI surface via inducing superconductivity by 
proximity effect. On the other, in experiment, 3D TI 
can be tuned into superconductor through doping with 
coppe r 29 ' 30 or applying high pressur o 31 ' 32 . Several theo- 
retical works have been done to analyze the superconduc- 
tivity in these TI materials 3 - 3 - - — . But these investigations 
are all about the 3D TI, and little attention has been 
paid to the superconductivity in QSH system (2D TI). 
More important, all these theoretical works start with 
the assumption of finite bulk superconductivity, which is 
suggested by experiments. But according to our knowl- 
edge, no discussion about the quantum phase transition 
between the topological insulating state and supercon- 
ducting state in TI has been presented so far, which is 
surely an intriguing theoretical issue. 

In this paper, we theoretically study the influence of 
the attractive interaction on the topological insulating 
state in the QSH system via self-consistent mean field 
method. We start with the Kane-Mele model of QSH sys- 
tem. The superconductivity is brought on by the attrac- 
tive Hubbard U interaction. With self-consistent mean 
field method, we investigate the phase transition from 
the topological insulating state to the superconducting 
state. Our results clearly show the existence of the edge 
superconducting state (ESS), in which the bulk is still 
insulating and superconductivity only appears near the 
edges. We point out that the phase transition from TI 
into superconducting state actually splits into two steps. 
Turning on the attractive U, TI will first evolve into the 
ESS immediately, and after exceeding a critical U c the 
whole system will become a superconductor. The rea- 
son is straightforward. TI has a bulk gap and gapless 
edge states. Electron pairs appear in the bulk only if the 
binding energy is larger than the bulk gap. From this 
point of view, the ESS seems to be a general property 
of the superconductivity in TI. Interestingly even if the 



FIG. 1: (Color online) Schematic illustration of the honey- 
comb lattice (a) and Zigzag ribbon structure (b) of the QSH 
system. 



FIG. 2: (Color online) (a) The zero temperature supercon- 
ducting gap A(U, Xso) in bulk for the half filling case, (b) U c 
in bulk as function of A so at half filling. 



bulk gap is zero, the ESS still occurs in the half filling 
case because of the special density of states of the Dirac 
fermion in ID (edge state) and 2D (bulk). We further 
deduce that the ESS also exists in 3D TI, but it has a 
critical U e d ge at half filling because of the special 2D den- 
sity of states. Finally, we give a short discussion about 
the effective model of ESS. 

Of course, we notice that the realistic phase transi- 
tion phenomena in the attractive U Kane-Mele-Hubbard 
model will be more complicated. For example, the possi- 
ble CDW phase, the possible topological superconductor 
phase with p wave pairing, and the BCS-BEC crossover 
are not involved in this work. But in this work we want to 
focus on the ESS which is a characteristic of the super- 
conductivity of TI and has not been noticed in former 
studies. We believe that our mean field study offers a 
good and reasonable starting point for further investiga- 
tion. 

The attractive U Kane-Mele-Hubbard model is defined 
as H = Hq + Hjj, where Hq is just the Kane-Mele model 
for the QSH system and Hjj describes the attractive Hub- 
bard U term which induces the superconductivity. We 
have 
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where <> indicates the nearest neigborhood (NN) hop- 
ping, t is the hopping amplitude and jj, is the chemical 
potential. The second term is the spin-orbit interaction. 
A so is the spin-orbit coupling constant, and <<>> de- 
notes the next nearest neighborhood (NNN) hopping, r 
is the Pauli matrix, uy = ±1 depending on the rela- 
tive orientation of the two bonds connecting i j sites 25 . 
Since the honeycomb lattice is a bipartite, it is conve- 
nient to split the lattice into two sublattice A and B. 
In the following, we use af (bf a ) denotes the creation 
operator in sublattice A (B). The Hubbard U term is 
Hjj = —U'^2 i ni^ni^ which is treated with mean field 
approximation (U > 0). In order to understand the su- 
perconducting states in QSH system well, we first inves- 
tigate the bulk properties in k space. Then, we consider 
the zigzag ribbon structure which concentrates on the 



edge states. Finally, we discuss the effective model of the 
ESS. 



With the relation ci 



Hamiltonian can be changed into momentum space. Na 
is the site number of sublattice A, and sublattice A 
and B are equivalent. The NN hopping is Hnn — 
-tJlkAlka^bka + ltb^a ka ), where 7fc = 7 (fc) = 
J2 s .e zkSi . £1=1,2,3 is the nearest neighborhood vectors 
[See Fig. 1 (a)]. For honeycomb lattice, Si = 00(5, -^), 

#2 = 00(51 — ^) and S 3 = a (— 1,0) . Here a is 
the lattice constant. The spin-orbit coupling is H so = 

£**k[ a fct°*t ~ atl a U ~ 6 fct 6fe t + h k 

>>/$, 
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J ki b ki\ where A fc 



2\ so [—sin(\/3aok y ) + 2cos(^aok x )sin(^-aoky)]. With 
mean field approximation, the negative Hubbard U term 
is Hu = - X) A [A A a^ot fe4r + A^a_ H a feT + A B b£ f b± kl + 

A* B b-kibk-f], with A A = j^J2k( a -kl a kt)- Since tne 
sublattice A and B are equivalent, we assume A a = 
Ab = A. In Nambu basis $^ = (ajj^, 
we have H = J2k ^t H k^k with 
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Diagonalizing Hk, the excitation energy of the Bogoli- 
ubov quasiparticles are E u= ±(k) = (e& ± y) 2 + A 2 
with efc = \/A?~+ t 2 \jk\ 2 and v is the band index. The 
gap equation is 
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The average electron density is 
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where n — N c /2Na and r) v= ± = ±1- Thus, given n 
and U , A and \i can be determined self-consistently with 
above equations^. Here we calculate the half filling case 
(n = 1) at zero temperature. The results are shown in 
Fig. 2. In Fig. 2 (a), we calculate the zero tempera- 
ture superconducting gap A as a function of U and A so . 
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cross section. H r n,bon(k x ) is a 4A x AN matrix 



FIG. 3: (Color online) (a) The zero temperature supercon- 
ducting gap A across the zigzag ribbon at half filling (N = 20, 
i.e. 40 sites across the ribbon), (b) Dashed line (blue): A(U) 
at the ribbon edge. Solid line (red) : A(U) inside the ribbon 
(middle site). 



We see that for any X so there exists a critical U c . Only 
when U > U c , A becomes nonzero. The U c here ac- 
tually reflect the competition between the formation of 
cooper pair and the bulk gap. Since there is a bulk gap 
in topological insulating state, there are no free carriers 
to start from. The attractive U can induce superconduc- 
tivity only if the binding energy is larger than the cost 
to produce free electrons and holes across the bulk gap. 
The superconductivity in such gapped system has been 
carefully discussed in semiconductor system^!. The U c is 
given in Fig. 2 (b) as a function of \ so . We know that 
A so determine the bulk gap in Kane-Mele model, where 
the larger A so the larger gap is. So U c should increase 
with A so since larger U c is needed to overcome the larger 
bulk gap. An interesting case is U c « 2.1t when A so = 0. 
ft indicates that even without bulk gap U c still exists at 
half filling. This phenomenon has been noticed in former 
study in optical lattice system^, ft results from the spe- 
cial density of states of the honeycomb lattice which will 
approach zero around the Dirac points. 

In topological insulating states, the only free carriers 
is just the gapless edge states. Beyond its linear dis- 
persion, the spin of the edge state are locked with its 
momentum. We study the zigzag ribbon structure so as 
to make clear the phase transition process of the edge 
states. The zigzag ribbon structure is shown in Fig. 1 
(b). It has infinite length in the longitudinal direction 
(x direction) and finite width in the transverse direction 
(y direction). The unit cell of the ribbon structure is 
labeled by integer indices m and n. Due to the transla- 
tional invariant, after Fourier transformation along the x 
direction, the tight binding Hamiltonian can be written 
as 

/dk 
-^■<t> + {kx)Hribbon{k x )(j){k x ) (4) 

where (f) + {k x ) = [■■■ ,a^(k x n),b^(k x n), ■ ■ ■ ,a±(k x n), 
b^(k x n),---} is the basis (47V vector) with n — 
1, 2, • • • , N. Here n is the row index and N is the width of 
ribbon, i.e. the number of unit cell along the transverse 
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where H a —^(k x ) are 2N x 2N tridiagonal matrices. De- 
tail of the expression is given in appendix. The attractive 
U interaction is still treated with mean field approxima- 
tion 

H v =- ^2[A An a+ (k x n)a+ (-k x n) 

nk x (6) 

+ A Bn b + (k x n)b + (-k x n)} + h.c 

where A An = j^J2k m (ai(-k x n)a^(k x n)} . A An (A Bn ) 
is the pairing potential on sublattice A (B) of row n. N r 
is the number of the unit cell of each row. Therefore, 
with the basis in Nambu representation ^ + (k x ) = 
[••• ,a^(k x n),b^(k x n), ■ ■ ■ , a±{-k x n), b±{-k x n), ■ ■ ■ ], 
the mean-field BCS Hamiltonian can be expressed as 
H mf = J ^ + (k x )H sc (k x )V{k x ) where 
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Here A^ is a diagonal matrix, the diagonal elements of 
which is (. . . , A An, A Bn , • • • ). Given A An, A s „ and the 
chemical potential (i.e. the average electron density n e = 
N e /[2N r ■ N]), H sc can be diagonalized numerically and 
we can get the energy dispersion and eigenfunction of the 
Bogoliubov quasiparticles. Thus, A An and A Bn can be 
determined self-consistently. One thing should be noticed 
that since the appearance of the ribbon edge, Aa 71 and 
A Bn are no longer equivalent in the calculation of ribbon 
structure. The results for half filling case are shown in 
Fig. 3. 

In Fig. 3 (a), we see that below U c the gap A in the 
bulk is zero, and A around the edge is nonzero. It means 
that the superconductivity only appears near the edges 
and the bulk is still insulating when U < U c . This is 
just the ESS (edge superconducting state) we mentioned 
above, which is the main results of this paper. Nor- 
mally, the mechanism of edge superconductivity mainly 
concerns the proximity effect which is not intrinsic. Here 
our results indicate the possibility of intrinsic ESS. Mean- 
while, we note that surface superconductivity in topo- 
logical flat band system has been studied recently^. It 
implies that the ESS is a general property of the system 
with topological protected edge states. When U > U c , 
superconductivity also appears in the bulk, but the gap 
A near the edges is still larger than that in the bulk. 

In Fig. 3 (b), we show the U dependence of the gap 
A at the edge and in the bulk respectively. Clearly, 
bulk superconductivity has a critical U c but edge super- 
conductivity has not. Small U can immediately induce 
nonzero edge superconductivity. It is because that ID 
Dirac fermion (linear dispersion of the edge state) has 
a constant density of state. As a comparison, we have 
demonstrated that the zero density of states of 2D Dirac 
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the band basis. We have 



FIG. 4: (Color online) (a) The self-consistent results of A((7) 
with the effective model of ESS (zero temperature). Dashed 
line (blue) :5N = 0. Solid line (red) : 8N = 0.15. We set 



hvf = 0.2t ■ a\ and momentum cutoff k c 



where t 



is the hopping amplitude and di = V3ao is the ID lattice 
constant of the tight binding model. The inner: mean field 
chemical potential n versus U for SN = 0.15. (b) Temperature 
dependence of A for half filling case. Dashed line (blue): 
U = It, Solid line (red): U = 1.5*. 



fermion around the Dirac points results in the finite U c 
of the bulk superconductivity when gap is zero at half 
filling. According to the discussion above, an interesting 
inference is that 2D ESS (surface superconducting state) 
also exist in 3D TI but with a finite U e d ge at half filing. 
Due to the appearance of the ESS in the attractive Kane- 
Mele-Hubbard model, the phase transition from topologi- 
cal insulating state to the superconducting state actually 
splits into two steps. Increasing the attractive U, the 
topological insulating state will immediately change into 
to ESS with any small U, and when U > U c , the the 
whole system becomes superconductor. We also calcu- 
late the slightly doping case (/j ^ but still in the bulk 
gap) and the results are qualitatively similar. 

Finally, We study the effective model of the ESS. 
The effective model of the helical edge states is Ha = 
hvf J dx^^id^af — ^^id x ^Li). The attractive in- 



teraction is H v = —U J cte$^ t $Rf$^$£,4.. Since the 
electron spin is locked with its momentum, we ignore 
the right-moving (left-moving) index R (L) in the follow- 
ing. In momentum space, we have H = f ^tp + H {k)ip 



where ip~* 



and 
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The eigenvalues are E u= ± — —fi±hvf\k\ with v = ± 
is the band index. Since not the spin but the helic- 
ity is good quantum number here, the upper (v = +) 
and lower (v — — ) bands correspond to different he- 
licity. Based on the BCS mean field approximation, 
Hu = -J f (A*c_ feiCfct +Ac+ c± ki ) with A = UJ f < 
C-kl c kt >■ It should be noticed that when k > (k < 0), 
< c-fc^Cfc-t- > indicate the pairing in upper band v = + 
(lower band v = — ). Thus, A includes superconducting 
pairs in both bands. 

It is convenient to express the superconducting Hamil- 
tonian in the Nambu basis ip + = [ci.,c_fej_] but not in 



H,sc 



( frvfk - 
\ -A* 



-A 



-hvfk 



fi 



The energy dispersion of the Bogoliubov quasiparticle is 
= \/A 2 + (hvjk — fi) 2 . We get the gap equation 

1 _ 1 C dfctanh[/3e B (fc)/2] 
U ~ 2 



2tt 



e B (k) 



In order to determine /x and A, we need the number 
equation concerning the particle conservation 
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where v 2 = ^ [1 



Tiv f k—[i i 



Jf is the usual Fermi function 



2 1" e B (k) 

and N- is the electron number of the filled band v — — . 
With above equations, we can determine the properties 
of the ESS with given U and SN. The gap A(U) is calcu- 
lated self-consistently for cases 8N = (half filling) and 
6N = 0.15 for example (See in Fig. 4). Since it is an 
effective model of lattice system, it is natural to use the 
hopping t and ID lattice constant a\ = V3ao as the unit. 
We can deduce the parameters, e.g. Vf, via fitting the 
tight binding band structure. In Fig. 4 (a), the results 
show that when U is rather small the superconducting 
gap A is still nonzero which is qualitatively consistent 
with the tight binding calculation. We also calculate the 
temperature dependence of the gap A in Fig. 4 (b). 

The calculations above are only on the mean field level. 
However, the influence of fluctuation can not be ignored 
since it is very important and will kill the superconduc- 
tivity in strictly ID system. The ESS we studied here 
can actually be viewed as a quasi-one-dimensional super- 
conducting system which is similar as the ultrathin su- 
perconducting nanowir o 41 ' 42 . A key feature of quasi-one- 
dimensional superconducting nanowire is that thermally 
activated phase slip and quantum phase slip processes 
will induce finite resistance when T < T c . Furthermore, 
because that the ESS is not an isolated ID system, it is 
possible that the coupling between the environments (e.g. 
the bulk or the substrate) may stabilize the supercon- 
ductivity, which is like the case of carbon nanotub o 43 ' 44 . 
The situations of ESS in real materials are complex and 
still an open question, but we propose that the concep- 
tion of ESS could be experimentally realized and exam- 
ined in atomic optical lattice system. Recently, various 
schemes to realize topological insulating state in optical 
lattice system have been proposed 4 ^—. And the attrac- 
tive U Hubbard model has been intensively studied in 
optical lattice system in order to investigate the BCS- 
BEC crossover^ - — . So producing topological insulating 
state with attractive interaction is straightforward in the 
optical lattice system . It means that it is possible to re- 
alize the conception of ESS in the optical lattice system. 

In summary, we investigate the phase transition 
from the topological insulating state to superconduct- 
ing state in attractive Kane-Mele-Hubbard model with 
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self-consistent mean field method. We clearly manifest 
the existence of the edge superconducting state which is 
superconducting at the edge but insulating in the bulk. 
The ESS results from the interplay between the attrac- 
tive interaction and the special energy band of TI. Thus, 
in contrast to the proximity effect induced edge super- 
conducting state in TI, the ESS we show here is intrinsic 
and is a general characteristic of the superconductivity 
of TI. In this model, increasing U, the ESS will occur im- 
mediately and when U > U c the whole system becomes 
superconductor. The critical U c of the bulk has been 
calculated. Due to the constant DOS of the edge state, 
there is no U c for the ESS. The effective model of the ESS 
has also been discussed. We propose that the conception 
of ESS could be experimentally realized in atomic optical 
system. 
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APPENDIX 



Here, we give the expression of Ha=fi in Eq. (5). 
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Here 



■k x 



2\ so sin{k x ai) 



and S' fcx = 
concerning the spin-orbit cou- 
pling; Xk = — t • (1 + e a, ° 1 ) is related with the next 
neighborhood hopping; r/ = +1(— 1) for spin up (down). 
ai = \/3ao is the ID lattice constant, i.e. the distance 
between adjacent unit cells along the x direction [See in 
Fig. 1 (b)]. 
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